Collaborative content evaluation

ABSTRACT

In an example embodiment, a system for evaluating published content is provided. The system includes at least one processor and a plurality of components including instructions executable by the at least one processor. The components include a reporting component and an evaluation component. The reporting component is to receive a communication from one of a plurality of users of a network-based system, the communication identifying content accessible via the network-based system as objectionable. An evaluation component is to evaluate the identified content based on a reputation value of the one of the plurality of users to determine an action to take with respect to the identified content. The reputation value is based at least in part on a history of objectionable content reporting by the one of the plurality of users.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/071,906, filed Mar. 16, 2016, which is a continuation of U.S. patentapplication Ser. No. 13/608,095, filed Sep. 10, 2012 (issued as U.S.Pat. No. 9,292,868), which is a continuation of U.S. patent applicationSer. No. 13/104,213, filed May 10, 2011 (issued as U.S. Pat. No.8,266,156), which is a continuation of U.S. patent application Ser. No.12/772,938, filed May 3, 2010 (issued as U.S. Pat. No. 7,966,335), whichis a continuation of U.S. patent application Ser. No. 11/646,888, filedDec. 28, 2006 (issued as U.S. Pat. No. 7,711,684). Each of theaforementioned applications is herein incorporated by reference in itsentirety.

TECHNICAL FIELD

This application relates to a method and system for evaluating contentdata.

BACKGROUND

As online applications mature, users and merchants increasinglycommunicate and participate in a variety of transactions and commercewith each other. Buyers and sellers (e.g., individuals and merchants)transact with each other based on good faith and whatever knowledge theymay have about each other as transacting parties and/or members of thetransacting community. However, as in any community, there may be usersthat attempt to cause other users harm or violate policies set forth bythe network-based administrator. For example, a user may try to defraudother users by misrepresenting a product in a listing, harm another userthrough malicious postings, etc. (e.g., reviews, etc.), or have unfairbusiness practices (e.g., inconsistent shipping charges).

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings in which:

FIG. 1 is a network diagram depicting a network system, according to oneembodiment, having a client-server architecture configured forexchanging data over a network;

FIG. 2A is a block diagram illustrating an example embodiment ofmultiple publication applications, which may be provided as part of anetwork-based publisher;

FIG. 2B is a block diagram illustrating an example embodiment of avarious modules that may be used to execute the processes describedherein;

FIG. 3 is a block diagram of an example embodiment illustrating a flowof four reports through the network system;

FIG. 4 is a flow chart 400 illustrating at a high level operations fordetermining a user reputation value, evaluating received content, andtaking action with respect to the content based on the user reputationvalue; and

FIG. 5 shows a diagrammatic representation of machine in the exampleform of a computer system within which a set of instructions may beexecuted causing the machine to perform any one or more of themethodologies discussed herein.

DETAILED DESCRIPTION

Although the present invention has been described with reference tospecific example embodiments, it will be evident that variousmodifications and changes may be made to these embodiments withoutdeparting from the broader spirit and scope of the invention.Accordingly, the specification and drawings are to be regarded in anillustrative rather than a restrictive sense.

In various embodiments, a system and method to generate user reputationvalues includes a network system (e.g., a network-basedcommunity/publisher) to determine a reputation value for one or moreusers of a network-based community, the reputation value being based onone or more user attributes associated with the one or more users.Content (e.g., a fraudulent listing) may then be evaluated based on thereputation value of the one or more users, the content being received inone or more communications from the respective one or more usersidentifying the content as objectionable within the network-basedcommunity. In one embodiment, the reputation value of a user may beadjusted by the network system based on validating the objectionablecontent.

In another embodiment, the network system receives a communication fromone or more users associated with a network-based community, thecommunication providing content identified by the one or more users asobjectionable within the network-based community. The system may thentake an action with respect to the content based on a reputation valueassociated with the one or more users.

FIG. 1 is a network diagram depicting a network system 100, according toone embodiment, having a client-server architecture configured forexchanging data over a network. For example, the network system 100 maybe a publication/publisher system where clients may communicate andexchange data with the network system 100. The data may pertain tovarious functions (e.g., online item purchases) and aspects (e.g.,managing content and user reputation values) associated with the networksystem 100 and its users. Although illustrated herein as a client-serverarchitecture for simplicity, other embodiments may include other networkarchitectures such as a peer machine in a peer-to-peer (or distributed)network environment.

A data exchange platform, in an example form of a network-basedpublisher 112, may provide server-side functionality, via a network 114(e.g., the Internet) to one or more clients. The one or more clients mayinclude users that utilize the network system 100 and more specifically,the network-based publisher 112, to exchange data over the network 114.These transactions may include transmitting, receiving (communicating)and processing data to, from, and regarding content and users of thenetwork system 100. The data may include, but is not limited to, contentand user data such as feedback data, user reputation values, userprofiles, user attributes, product and service reviews, product,service, manufacture, and vendor recommendations and identifiers,product and service listings associated with buyers and sellers, auctionbids, etc.

In various embodiments, the data exchanges within the network system 100may be dependent upon user selected functions available through one ormore client/user interfaces (UIs). The UIs may be associated with aclient machine, such as a client machine 120 utilizing a web client 116.The web client 116 may be in communication with the network-basedpublisher 112 via a web server 126. The UIs may also be associated witha client machine 122 utilizing a client application 118, or a thirdparty server 140 hosting a third party application 138. It can beappreciated in various embodiments the client machine 120, 122, or thirdparty application 138 may be associated with a buyer, a seller, paymentservice provider or shipping service provider, each in communicationwith the network-based publisher 112 and optionally each other. Thebuyers and sellers may be any one of individuals, merchants, serviceproviders, etc.

Turning specifically to the network-based publisher 112, an applicationprogram interface (API) server 124 and a web server 126 are coupled to,and provide programmatic and web interfaces respectively to, one or moreapplication servers 128. The application servers 128 host one or morepublication application (s) 132. The application servers 128 are, inturn, shown to be coupled to one or more database server(s) 134 thatfacilitate access to one or more database(s) 136.

In one embodiment, the web server 126 and the API server 124communicates and receives data pertaining to a user reportingobjectionable content (e.g., fraudulent listing, overpriced shipping,etc.) via various user input tools. For example, the web server 126 maysend and receive data to and from a toolbar or webpage on a browserapplication (e.g., web client 116) operating on a client machine (e.g.,client machine 120). The API server 124 may send and receive data to andfrom an application (e.g., client application 118 or third partyapplication 138) running on another client machine (e.g., client machine122 or third party server 140).

The publication application(s) 132 may provide a number of publisherfunctions and services (e.g., listing, payment, etc.) to users thataccess the network-based publisher 112. For example, the publicationapplication(s) 132 may provide a number of services and functions tousers for reviewing and providing feedback about transactions andassociated users. Additionally, the publication application(s) 132 maydetermine a reputation value for one or more users of a network-basedcommunity, the reputation value being based on one or more userattributes associated with the one or more users and content (e.g.,fraudulent listing) may then be evaluated based on the reputation valueof the one or more users, the content being received in a communicationfrom the one or more users identifying the content as objectionablewithin the network-based community (e.g., network system 100). Thepublication application(s) 132 may then adjust the reputation value of auser based on validating the content as objectionable within the system.Additionally, the publication application(s) 132 may take an action withrespect to the content identified as objectionable based on a reputationvalue associated with the one or more users.

In various embodiments, rules may be applied to determine when contentflagged as objectionable is at least one of reported to a systemadministrator (e.g., a customer service representative), removed withoutreporting, provided to one or more other reputable users for evaluation,or any combination thereof.

In one embodiment, the publication application(s) 132 may provide a uservia a client (e.g., web client 116) with an interface that includesinput fields for reporting objectionable content that may be posted orotherwise associated with the network-based publisher 112. For example,the user may determine an item or information listed on thenetwork-based publisher 112 is a fraudulently represented item, an itemcontrary to policy (e.g., listing of a body part, etc.), contentattempting to damage a reputation of another user, or content pertainingto unfair or bad business practices (e.g., inconsistent or overpricedshipping). The user may then communicate to the network-based publisher112 the objectionable content pertaining to the listing and/orobjectionable conduct and associated content of another user forevaluation and action (e.g., internally or by the community of users).For simplicity hereafter, the reporting of objectionable content alsoimplies a reporting of the associated objectionable conduct of the oneor more users associated with the objectionable content.

FIG. 1 also illustrates a third party application 138 that may executeon a third party server 140 and may have programmatic access to thenetwork-based publisher 112 via the programmatic interface provided bythe API server 124. For example, the third party application 138 mayutilize information retrieved from the network-based publisher 112 tosupport one or more features or functions on a website hosted by thethird party. The third party website may, for example, provide one ormore objectionable content reporting mechanisms, feedback, socialnetworking, publisher or payment functions that are supported by therelevant applications of the network-based publisher 112.

FIG. 2A is a block diagram illustrating an example embodiment ofmultiple publication application(s) 132, which are provided as part ofthe network-based publisher 112. The network-based publisher 112 mayprovide a multitude of reporting, feedback, reputation, socialnetworking, and listing and price-setting mechanisms whereby a user mayreport objectionable content and possibly associated conduct found onthe network-based publisher 112, or may be a seller or buyer that maylist or buy goods and/or services (e.g., for sale) published on thenetwork-based publisher 112.

The publication application(s) 132 are shown to include one or moreapplication(s) which support the network-based publisher 112, and morespecifically, the generation of reputation values for users based onuser attributes, the reporting of objectionable content by those users,and subsequent action by the network-based publisher 112 based on thereported content and the reputation value of the user(s).

Store application(s) 202 may allow sellers to group their listings(e.g., goods and/or services) within a “virtual” store, which may bebranded and otherwise personalized by and for the sellers. Such avirtual store may also offer promotions, incentives and features thatare specific and personalized to a relevant seller. In one embodiment, aseller using a virtual store to sell their goods and services may resultin the network-based publisher 112 determining a higher reputation valuebecause of an inherent trustworthiness (e.g., higher reputation value)of a “business” over an individual seller.

Feedback application(s) 204 may allow parties that transact utilizingthe network-based publisher 112 to establish, build, and maintainbuyer/seller reputations, which may be made available and published topotential trading partners (e.g., users of the network-based publisher112). Consider, for example, where the network-based publisher 112supports person-to-person trading, users may have no history or otherreference information whereby the trustworthiness and/or credibility ofpotential trading partners may be assessed. The feedback application(s)204 may allow a first user, for example through feedback provided byother users, to establish a buyer/seller reputation within thenetwork-based publisher 112 over time and transactions. Thus, otherpotential transaction partners (users) may then reference thebuyer/seller reputation value of the user for the purpose of assessingcredibility, trustworthiness, etc.

It can be appreciated by one skilled in the art that users may have amultitude of various types of reputation values in the network-basedpublisher 112. For example, a user may have a reputation value for beinga buyer, one as a seller, and another as an objectionable contentidentifier. Additionally, the reputation value types may have an impacton one another. For example, a user having a high buyer/sellerreputation value may as a result have a higher reputation value foridentifying objectionable content than someone with a very low ornegative buyer/seller reputation value. However, for simplicity, areputation value as used hereafter will refer primarily to a valueassociated with a user's accuracy for identifying objectionable content.

In an example embodiment, the network-based publisher 112 may includereview and recommendation application(s) 206. Social networkingapplication(s) 208 may work in conjunction with the review andrecommendation application(s) 206 to provide a user interface tofacilitate the entry of content such as reviews and recommendations ofproducts and services to other users or communities/groups of users. Areview may be a text entry of the community group member's opinion, astandard review form including check boxes indicating a levelsatisfaction, or a combination of both, etc.

Recommendations may include a specific type of item, a specific brand orservice for a type of item, a specific retailer for the item, etc. Inone embodiment, providing a community service, such as reviews andrecommendations to the network-based publisher 112 may result in aperception of a higher reputation or rating in the community (e.g., moretrustworthy). This may be recognized by a weighting function determininga higher weighted value to calculate the first user's overall reputationvalue.

Social networking application(s) 208 may support social networkingfunctions, including building and maintaining community groups joined orcreated by a user. For example, one of the social networking functionsmay be used to determine a user attribute (e.g., numerical value)associated with a user creating, belonging, and participating in acommunity group. A user may have a higher rating in the community (e.g.,more trustworthy) based on community participation and association(e.g., moderator, review publisher, etc.). The higher rating may thenresult in the determination of a higher reputation value over a user notparticipating in the community through group associations, leadership,etc.

In one embodiment, content reporting application(s) 210 may be used toreceive content communicated in an example form of a report to thenetwork-based publisher 112 via a client machine (e.g., client machine120). The content reporting application(s) 210 may provide data to theweb server 126 (e.g., interface data) and the API server 124 tofacilitate the communicating of the report of objectionable content. Inaddition to receiving the report, the content reporting application(s)210 may facilitate the retrieval of user information corresponding tothe user communicating the report. For example, user name and reputationvalue from the database(s) 136 through the database server(s) 134.

In one embodiment, reputation application(s) 212 may work in conjunctionwith the content reporting application(s) 210 to determine one or morereputation values associated with a user in the network system 100. Asdescribed above, this may be based on one or more user attributes. Theuser attributes may include, but are not limited to, a history ofaccurately reporting objectionable content, a reputation value of a uservalidating a report originating from another user, a current reputationvalue, a value of the transaction, prior transaction values associatedwith the user, prior transaction values associated with one or moreother users, number of prior transactions of the first user, number ofprior transactions of the one or more other users, group association ofa user, community service and participation (e.g., writes reviews,etc.), an imported reputation value, and a category of transactionincluding a user's expertise (e.g., power seller) in a category.Additionally, some or all of these attributes may have values or weightsassigned to them that may be used in conjunction with other values andfactors to determine a reputation value of a user.

After a user reports content via the content reporting application(s)210, the network-based publisher 112 may then take action with respectto the reporting user or users and the reported content. The actiontaken with respect to the reported content and the user may be based onthe reputation value of the user and optionally other users.

Additionally, the content reporting application(s) may apply one or morerules to determine when content is flagged as objectionable. These rulesmay include at least one of reporting the content to a systemadministrator (e.g., a CSR (customer service representative)) forinvestigation and confirmation, removing the content without reportingit to a CSR, providing it to one or more other reputable users forevaluation, or any combination thereof. For example, it may take threeusers of a medium reputation or just one user of a high reputation toidentify a listing as fraudulent resulting in the listing beingautomatically pulled from the network-based publisher 112. In anotherexample, a low reputation user reporting a listing as fraudulent may beoverridden by one highly reputable user indicating the listing is notfraudulent, resulting in no action taken with respect to the listing.

FIG. 2B is a block diagram illustrating an example embodiment of areporting module 250, an evaluation module 252, a reputation module 254,and a validation module 256, which may be utilized by the contentreporting application(s) 210 and the reputation application(s) 212 toreceive a report of objectionable content, determine or adjust areputation value of users, validate a report, and evaluate and takeaction with respect to the reported content and reporting user.

In one embodiment, the reporting module 250 may receive a communicationfrom one or more users reporting objectionable content. The reportingmodule 250 may then retrieve additional data associated with eachreporting user from the database(s) 136. In another embodiment, thereporting module 250 retrieves embedded user data from each communicatedreport.

The evaluation module 252 may evaluate the communication containing theobjectionable content received from the user(s). In one embodiment, theevaluation module 252 works in conjunction with a reputation module 254to retrieve user attributes, such as the current reputation value of theuser(s), if any. Additionally, the evaluation module 252 may categorizethe reported content by various methods, such as a keyword analysis,manual entry (e.g., by a CSR), and by reading a field in thecommunication that includes a user entry indicating the type of content.Then based on the current reputation value, the evaluation module 252determines how to treat the reported content. Examples of this treatmentare illustrated and discussed in further detail with reference to FIG.3.

The validation module 256 may work in conjunction with the evaluationmodule 252 and reputation module 254 to determine if the reportedcontent is objectionable and based on that determination how todetermine or adjust the reporting user(s) reputation value, if at all.Examples of the determining or adjusting of the reputation value areillustrated and discussed in further detail with reference to FIG. 3.Additionally, in various embodiments the validation module 256 may, withrespect to the objectionable or flagged content, report it to a systemadministrator (e.g., a CSR for confirmation), remove it withoutreporting to a CSR, provide it to one or more other reputable users forfurther evaluation, or any combination thereof, as discussed above.

FIG. 3 is a block diagram of an example embodiment illustrating a flowof four reports through the network system 100. On the client side,there are four users representing four clients of the network system 100(e.g., four client machines 120,122), the users reporting content Athrough D, respectively, and each user has a different reputation value.User 302 has a reputation value of 1000, user 304 has a reputation valueof 500, user 306 has a reputation value of 10, and user 308 has areputation value of 1.

In one embodiment, the network-based publisher 112 receives, evaluates,and takes action with respect to the reported content based on a user'sreputation value. In some embodiments, based on the validation of thecontent by the network-based publisher 112, a new or adjusted reputationvalue is determined for the reporting user(s). In other embodiments, theevaluation and action taken with respect to the reported content isbased on a user's reputation value exceeding, falling below, or fallingwithin a range of upper and lower threshold values. For example, contentreported by a user having a reputation value of greater than a thresholdvalue of 900 may be automatically removed. The content reported by auser having a value above 100 but less than 900 may result in thecontent being transferred for validation or require concurrent reportingby one or more other users. The content reported by a user having avalue less than 10 may result in the content being ignored unlessconcurrently validated by some combination of fewer users with highreputation values or many users with low reputation values.

In various embodiments, rules are created to determine action to takewith respect to the content based on varying combinations of reportingusers. For example, as discussed above, it may take three users of amedium reputation (e.g., user 304 of reputation 500) or just one user ofa high reputation (e.g., user 302 of reputation 1000) to identify alisting as fraudulent resulting in the listing being automaticallypulled from the network-based publisher 112. In another example, a lowreputation user (e.g., user 306 of reputation value 10) reporting alisting as fraudulent may be overridden by one highly reputable user(e.g., user 302 of reputation 1000) indicating the listing is notfraudulent. It can be appreciated the threshold values and usercombinations in taking action and validating are configurable within thenetwork system 100.

Additionally, in various embodiments, the evaluation may includeanalyzing additional attributes associated with the reported content.For example, other content attributes may include, but are not limitedto category and risk value. A category of jewelry having a risk value of$100,000 USD may be treated differently (e.g., automatically referred toan administrator, such as a CSR) than a cookie recipe having a value of5.00 USD, which may be automatically removed depending on thecreditability of the reporting user as assessed by the reputation value.

In the first example, user 302 reports content A. At block 310 thesubmission of the content is evaluated. In various embodiments, this mayinclude retrieving user attributes (e.g., current reputation value,etc.) associated with the user 302 and categorizing the type of content.In one embodiment, the categorizing may be used in conjunction with theuser's 302 reputation value to determine a course of action to take withrespect to the content. For example, if the user 302 has a strongbuyer/seller reputation in the category associated with the report thenetwork-based publisher 112 at block 312 may automatically remove thecontent without consuming additional resources of the network-basedpublisher 112. However, if in a category outside of the expertise of theuser 302 a different course of action may be taken, such as validatingthe content (as objectionable) at block 314.

In one embodiment, the course of action may be to transfer the reportedcontent as a “case” to an administrator, such as a CSR. The contentplacement in the CSR queue may be directly proportional to the user's302 reputation value. In another embodiment, the administrator(s) may beone or more other users. The content may be communicated to the one ormore other users who may have an expertise or familiarity in thecategory associated with the reported content A. In the examplesdescribed herein, validating may also include checking if other usershave also reported the same content and taking action accordingly andevaluation may also include factoring in other content attributes (e.g.,risk value, category, etc.) as described above. In any case, ifvalidated at block 314 the reputation value of the user 302 may beincreased and the content removed at block 316. If not validated,because the user 302 has a reputation value of 1000, the network-basedpublisher 112 may consider the report a mistake and elect to notdecrease the reputation value of the user 302 at block 318.

In the next example embodiment, user 304 has a reputation value of 500and reports content B to the network-based publisher 112. After thecontent B is evaluated at block 320, in this embodiment, based on thereputation value of 500 the content is transferred to the top of a CSRqueue at block 322. In another embodiment, as described above, thecontent may be transferred to a user having a higher reputation value inthe category of the content for validation instead of the CSR. Ifvalidated, at block 324, the reputation value of the user 304 may beincreased and the content removed at block 326. If not validated, thenthe reputation value of the user 304 may be decreased at block 328 andthe content left on the system.

In the next example embodiment, user 306 has a reputation value of 10and reports content C to the network-based publisher 112. After thecontent C is evaluated at block 330, based on the reputation value of100, the content is transferred to the CSR queue according to standardpriority at block 332. As discussed above, the priority may be based onfirst in first out or may be dependent upon other content attributes,such as category or risk value. In another embodiment, as describedabove, the content may be transferred to a user(s) having a higherreputation value in the category of the content for validation insteadof a CSR. If validated, at block 334, the reputation value of the user306 may be increased and the content removed at block 336. If notvalidated, then the reputation value of the user 306 may be decreased atblock 338 and the content left on the system.

In the next example embodiment, user 308 has a reputation value of 1 andreports content D to the network-based publisher 112. After the contentD is evaluated at block 330, in this embodiment, based on the reputationvalue of 1 the content is validated at block 342. As discussed above,this may include being concurrently validated by other users reportingcontent D or by the content being transferred to a user having a highreputation value in the category. Again, the evaluation may also dependon assessing other content attributes as described above. If validated,at block 344, the reputation value of the user 308 may be increased andthe content removed. If not validated, then at block 346 the reputationvalue of the user 306 may be decreased, if possible, and the contentleft on the system.

FIG. 4 is a flow chart illustrating at a high level operations fordetermining a user reputation value, evaluating received content, andtaking action with respect to the content based on the user reputationvalue. At operation 402, reputation values for one or more users aredetermined based on one or more user attributes associated with eachuser (e.g., buyer/seller rating, history of identifying fraudulentcontent, etc.). At operation 404, a communication is received from oneor more users identifying problematic content within the network-basedcommunity (e.g., fraudulent content, overpriced shipping, etc.) Theproblematic content is evaluated, at operation 406, based on thereputation values associated with each of the one or more users andoptionally additional content attributes, such as category of contentand risk value of content (e.g., a $100,000 USD fraudulent listing). Atoperation 408, the problematic content is processed based on theevaluation. For example, as discussed above, the content may be removed,transferred to a CSR and/or another for validation, or left on thesystem (no action taken). Lastly, at operation 410, the one or more ofthe user's reputation values may be determined (e.g., created oradjusted) based on the processing of the problematic content.

FIG. 5 shows a diagrammatic representation of machine in the exampleform of a computer system 500 within which a set of instructions may beexecuted causing the machine to perform any one or more of themethodologies discussed herein. In alternative embodiments, the machineoperates as a standalone device or may be connected (e.g., networked) toother machines. In a networked deployment, the machine may operate inthe capacity of a server or a client machine in server-client networkenvironment, or as a peer machine in a peer-to-peer (or distributed)network environment. The machine may be a personal computer (PC), atablet PC, a set-top box (STB), a Personal Digital Assistant (PDA), acellular telephone, a web appliance, a network router, switch or bridge,or any machine capable of executing a set of instructions (sequential orotherwise) that specify actions to be taken by that machine. Further,while only a single machine is illustrated, the term “machine” shallalso be taken to include any collection of machines that individually orjointly execute a set (or multiple sets) of instructions to perform anyone or more of the methodologies discussed herein.

The example computer system 500 includes a processor 502 (e.g., acentral processing unit (CPU), a graphics processing unit (GPU) orboth), a main memory 504 and a static memory 506, which communicate witheach other via a bus 508. The computer system 500 may further include avideo display unit 510 (e.g., a liquid crystal display (LCD) or acathode ray tube (CRT)). The computer system 500 also includes analphanumeric input device 512 (e.g., a keyboard), a user interface (UI)navigation device 514 (e.g., a mouse), a disk drive unit 516, a signalgeneration device 518 (e.g., a speaker) and a network interface device520.

The disk drive unit 516 includes a machine-readable medium 522 on whichis stored one or more sets of instructions and data structures (e.g.,software 524) embodying or utilized by any one or more of themethodologies or functions described herein. The software 524 may alsoreside, completely or at least partially, within the main memory 504and/or within the processor 502 during execution thereof by the computersystem 500, the main memory 504 and the processor 502 also constitutingmachine-readable media.

The software 524 may further be transmitted or received over a network526 via the network interface device 220 utilizing any one of a numberof well-known transfer protocols (e.g., HTTP).

While the machine-readable medium 522 is shown in an example embodimentto be a single medium, the term “machine-readable medium” should betaken to include a single medium or multiple media (e.g., a centralizedor distributed database, and/or associated caches and servers) thatstore the one or more sets of instructions. The term “machine-readablemedium” shall also be taken to include any medium that is capable ofstoring, encoding or carrying a set of instructions for execution by themachine and that cause the machine to perform any one or more of themethodologies of the present invention, or that is capable of storing,encoding or carrying data structures utilized by or associated with sucha set of instructions. The term “machine-readable medium” shallaccordingly be taken to include, but not be limited to, solid-statememories, optical media, and magnetic media.

The Abstract of the Disclosure is provided to comply with 37 C.F.R. §1.72(b), requiring an abstract that will allow the reader to quicklyascertain the nature of the technical disclosure. It is submitted withthe understanding that it will not be used to interpret or limit thescope or meaning of the claims. In addition, in the foregoing DetailedDescription, it can be seen that various features are grouped togetherin a single embodiment for the purpose of streamlining the disclosure.This method of disclosure is not to be interpreted as reflecting anintention that the claimed embodiments require more features than areexpressly recited in each claim. Rather, as the following claimsreflect, inventive subject matter lies in less than all features of asingle disclosed embodiment. Thus the following claims are herebyincorporated into the Detailed Description, with each claim standing onits own as a separate embodiment.

What is claimed is:
 1. One or more computer storage media storingcomputer-usable instructions that, when used by one or more processors,cause the one or more processors to perform operations comprising:receiving a communication from a first user of a plurality of users of anetwork system, the communication identifying content accessible via thenetwork system as objectionable; evaluating the identified content basedon a reputation value of the first user to determine an action to takewith respect to the identified content, the reputation value being basedat least in part on a history of objectionable content reporting by thefirst user; taking the action with respect to the identified contentbased on the evaluation of the identified content; and adjusting thereputation value of the first user based on the action taken withrespect to the identified content.
 2. The computer storage media ofclaim 1, wherein the identified content comprises a product listing, aservice listing, a product review, or a service review.
 3. The computerstorage media of claim 1, the reputation value further being based on anexpertise level of the first user relating to the identified content. 4.The computer storage media of claim 1, the reputation value furtherbeing based on a level of participation of the first user with thenetwork system.
 5. The computer storage media of claim 1, the evaluatingof the identified content further being based on a content attributeassociated with the identified content.
 6. The computer storage media ofclaim 5, the content attribute comprising an item category associatedwith the identified content.
 7. The computer storage media of claim 1,the operations further comprising: receiving a plurality of additionalcommunications from the first user, each additional communication of theplurality of additional communications identifying corresponding contentaccessible via the network system as objectionable; validating eachinstance of the corresponding content to determine whether thecorresponding content is objectionable; and determining the reputationvalue of the first user based on an accuracy for identifyingobjectionable content based on validation of whether the correspondingcontent is objectionable.
 8. The computer storage media of claim 1,wherein taking the action with respect to the identified contentcomprises removing the identified content from publication based on thereputation value of the first user exceeding a threshold.
 9. Thecomputer storage media of claim 1, wherein taking the action withrespect to the identified content comprises removing the identifiedcontent from publication based on the reputation value of the first userfalling below a threshold and a predetermined number of other users ofthe plurality of users having identified the content as objectionable.10. The computer storage media of claim 1, wherein taking the actionwith respect to the identified content comprises referring theidentified content to an individual associated with the network systembased on the reputation value of the first user falling below athreshold, and removing the identified content from publication based onthe individual validating the identified content as objectionable, theindividual comprising a representative of the network system.
 11. Thecomputer storage media of claim 1, wherein taking the action withrespect to the identified content comprises referring the identifiedcontent to an individual associated with the network system based on thereputation value of the first user falling below a threshold, andremoving the identified content from publication based on the individualvalidating the identified content as objectionable, the individualcomprising a second user of the plurality of users, the second userhaving a reputation value that is higher than the reputation value ofthe first user.
 12. A computer-implemented method comprising: receivinga communication from a first user of a plurality of users of a networksystem, the communication identifying content accessible via the networksystem as objectionable; evaluating the identified content based on areputation value of the first user to determine an action to take withrespect to the identified content, the reputation value being based atleast in part on a history of objectionable content reporting by thefirst user; taking the action with respect to the identified contentbased on the evaluation of the identified content; and adjusting thereputation value of the first user based on the action taken withrespect to the identified content.
 13. The method of claim 12, theevaluating of the identified content further being based on a contentattribute associated with the identified content.
 14. The method ofclaim 12, wherein taking the action with respect to the identifiedcontent comprises removing the identified content from publication basedon the reputation value of the first user exceeding a threshold.
 15. Themethod of claim 12, wherein taking the action with respect to theidentified content comprises removing the identified content frompublication based on the reputation value of the first user fallingbelow a threshold and a predetermined number of other users of theplurality of users having identified the content as objectionable. 16.The method of claim 12, wherein taking the action with respect to theidentified content comprises referring the identified content to anindividual associated with the network system based on the reputationvalue of the first user falling below a threshold, and removing theidentified content from publication based on the individual validatingthe identified content as objectionable, the individual comprising arepresentative of the network system.
 17. The method of claim 12,wherein taking the action with respect to the identified contentcomprises referring the identified content to an individual associatedwith the network system based on the reputation value of the first userfalling below a threshold, and removing the identified content frompublication based on the individual validating the identified content asobjectionable, the individual comprising a second user of the pluralityof users, the second user having a reputation value that is higher thanthe reputation value of the first user.
 18. A computer systemcomprising: a reporting component comprising at least one of one or moreprocessors of a computing device configured to receive a communicationfrom a first user of a plurality of users of a network system, thecommunication identifying content accessible via the network system asobjectionable; an evaluation component comprising at least one of theone or more processors of the computing device configured to evaluatethe identified content based on a reputation value of the first user todetermine an action to take with respect to the identified content, thereputation value being based at least in part on a history ofobjectionable content reporting by the first user; and a validationcomponent comprising at least one of the one or more processors of thecomputing device configured to take the action with respect to theidentified content based on the evaluation of the identified content,and to adjust the reputation value of the first user based on the actiontaken with respect to the identified content.
 19. The system of claim18, wherein the action taken with respect to the identified contentcomprises removing the identified content from publication based on thereputation value of the first user exceeding a threshold.
 20. The systemof claim 18, wherein the action taken with respect to the identifiedcontent comprises removing the identified content from publication basedon the reputation value of the first user falling below a threshold andat least one other user having identified the content as objectionable.